Perform Regression Testing Research Articles

Search-based detection of code changes introducing performance regression

• Detecting code changes responsible for performance regression is time-consuming. • Dynamic analysis is a credible source in detection performance regression but expensive. • Detecting performance regression problem formulated as optimization problem. • Detection rules generated by using a multi-objective evolutionary algorithm. In contemporary software development, developers commonly conduct regression testing to ensure that code changes do not affect software quality. Conducting performance regression testing after every code change is known to be expensive which emigres the need to direct the performance regression testing efforts on code changes that are most likely introducing performance regression. In this paper, we exploit code change metrics to identify the group introducing the regression based on a pre-trained detection rule. We present PRICE approach as a new formulation of detecting code changes introducing performance regression as an optimization problem using multi-objective evolutionary algorithms. PRICE evaluated using a set of 8000 commits, extracted from the Git project. Results show the effectiveness of our approach in accurately detecting performance regression introducing code changes. The average regression detection PRICE provides is 77% which is 22% more than the state-of-the-art deterministic approach. This improvement didn’t compromise the detection of the contradicted code changes that are not introducing a regression. PRICE provides detection of code changes not introducing a regression averaged 63%, which is also an improvement of 14% than the comparative approach. Using PRICE, we were able to explore new search spaces and provide competing results.

Towards a Shift in Regression Test Suite Development Approach in Agile

Objective: From the literature review, it is evident that the concept of “regression testing” inherited in agile software testing originates from software maintenance practices. Therefore, the existing algorithms for regression testing revolve around the software maintenance principles rather than agile methodology. The objective of this paper is to evaluate the degree of fitness of the existing regression test-suite development algorithms for performing the regression testing in agile. Methods: This paper performs a systematic literature review for research work published from 2006 to 2018, which includes survey of the existing regression testing algorithms to identify and overcome the challenges associated with them while performing regression testing in agile. This research paper considers the four research questions into scope for analyzing the fitness of existing regression test-suite development algorithm for performing regression testing under agile methodology. Further, this paper attempts to propose approach for the development of the regression test-suite suitable for regression testing under agile methodology. Results: The current regression test-suite development algorithm were found unsuitable for performing the regression testing under agile methodology due to the newly identified four key challenges associated with them. Conclusion: The current regression test-suite development algorithms aligned with software maintenance principles rather than agile methodology. In addition, the newly proposed approach for regression test-suite development found to be easily adaptable by agile teams as it aligns with agile methodology principles. Finally, this paper recommends the adoption of agile principle through the newly proposed approach for developing regression test-suite for performing regression testing under agile methodology.

Applying test case prioritization to software microbenchmarks

Regression testing comprises techniques which are applied during software evolution to uncover faults effectively and efficiently. While regression testing is widely studied for functional tests, performance regression testing, e.g., with software microbenchmarks, is hardly investigated. Applying test case prioritization (TCP), a regression testing technique, to software microbenchmarks may help capturing large performance regressions sooner upon new versions. This may especially be beneficial for microbenchmark suites, because they take considerably longer to execute than unit test suites. However, it is unclear whether traditional unit testing TCP techniques work equally well for software microbenchmarks. In this paper, we empirically study coverage-based TCP techniques, employing total and additional greedy strategies, applied to software microbenchmarks along multiple parameterization dimensions, leading to 54 unique technique instantiations. We find that TCP techniques have a mean APFD-P (average percentage of fault-detection on performance) effectiveness between 0.54 and 0.71 and are able to capture the three largest performance changes after executing 29% to 66% of the whole microbenchmark suite. Our efficiency analysis reveals that the runtime overhead of TCP varies considerably depending on the exact parameterization. The most effective technique has an overhead of 11% of the total microbenchmark suite execution time, making TCP a viable option for performance regression testing. The results demonstrate that the total strategy is superior to the additional strategy. Finally, dynamic-coverage techniques should be favored over static-coverage techniques due to their acceptable analysis overhead; however, in settings where the time for prioritzation is limited, static-coverage techniques provide an attractive alternative.

A Systematic Analysis of Regression Test Case Selection: A Multi-Criteria-Based Approach

In applied software engineering, the algorithms for selecting the appropriate test cases are used to perform regression testing. The key objective of this activity is to make sure that modification in the system under test (SUT) has no impact on the overall functioning of the updated software. It is concluded from the literature that the efficacy of the test case selection solely depends on the following metrics, namely, the execution cost of the test case, the lines of the code covered in unit time also known as the code coverage, the ability to capture the potential faults, and the code modifications. Furthermore, it is also observed that the approaches for the regression testing developed so far generated results by focusing on one or two parameters. In this paper, our key objectives are twofold: one is to explore the importance of the role of each metric in detail. The secondary objective is to study the combined effect of these metrics in test case selection task that is capable of achieving more than one objective. In this paper, a detailed and comprehensive review of the work related to regression testing is provided in a very distinct and principled way. This survey will be useful for the researchers contributing to the field of regression testing. It is noteworthy that our systematic literature review (SLR) included the noteworthy work published from 2007 to 2020. Our study observed that about 52 relevant studies focused on all of the four metrics to perform their respective tasks. The results also revealed that about 30% of the different categories of regression test case reported the results using metaheuristic regression test selection (RTS). Similarly, about 31% of the literature reported results using the generic regression test case selection techniques. Most of the researchers focus on the datasets, namely, Software-Artefact Infrastructure Repository (SIR), JodaTime, TreeDataStructure, and Apache Software Foundation. For validation purpose, following parameters were focused, namely, the inclusiveness, precision, recall, and retest-all.

Automation-as-a-Service (AaaS) for IDPA

Abstract: The Dell EMC Integrated Data Protection Appliance (IDPA) provides simplified configuration and integration of data protection components in a consolidated solution. As engineers works with different appliance variants, need to perform regression testing on all variants to check code stability. AaaS provides engineers with an application which will be independent of environment and can handle multiple operations at a time. All the scenarios will run as a service in a container at the same time and consolidated results will be provided back to user on UI. This will reduce manual effort with respect to manual execution and setting up environment. Keywords: IDPA, Regression testing, Appliance Configuration Manager, Automation-as-a-Service (AaaS), Swarm Intelligence

Comparative study of regression testing tools feature on unit testing

Regression testing is one of the essential activities in software development. Regression testing ensures that the software is still working properly and new software modifications do not cause unexpected defects. Most of the software industry carries out regression testing on the system under test (SUT) by utilizing tools automatically rather than being done manually which requires a long time estimation and ineffective process. Currently, there are many regression testing tools and variations that make software practitioners need additional time and money to analyze the capabilities of the tools. This paper is a comparative study of the capabilities of tools in carrying out the regression testing process in the java programming language. Each tool has experimented with user input data in the form of a SUT program and a test case that has the same characteristics. Furthermore, the tool’s performance is observed based on the feature evaluation criteria. The criteria used are support platform and technology, test case selection, and functionality. The results showed the similarities and differences in the features of the selected testing tool in performing regression testing. Thus it can help users to choose regression testing tools that suit their needs.

Test scenario prioritization from user requirements for web-based software

User requirements are the building blocks for development of software applications. User requirements decide the width and breadth of any software. Nowadays, test scenarios are prepared from the user requirements, which give the test engineers ample scope to review the test plan thoroughly before doing software testing. Regression testing is carried out to know the effect of requirement changes on the functionalities and performance of the software. Test scenario prioritization, which is one of the techniques to perform regression testing, maximizes the ease of debugging for the system under test. The code based test scenario prioritization and model based test scenario prioritization have their own limitations. So, to achieve ease of debugging and to get more time for reviewing the test plans, researchers are now working on test scenario prioritization using requirements collected from the end users. In this paper, we propose an approach named Requirement based test scenario prioritization to prioritize test scenarios using requirements collected from end users for developing software applications. The user’s functional requirements are collected and is assigned with some weight depending upon different factors like complexity of implementing the requirements, type of release of the requirements, requirement volatility, coupling between requirements etc. Test scenarios are generated from requirements collected from the end users. Then, the final priority weight of each test scenario is found out by considering the weight of each requirement covered by the corresponding test scenario and the percentage of requirements coverage made by each test scenario. The test scenarios are prioritized based on the final priority weight. The proposed approach is evaluated using average percentage of fault detection metric and is found to be very efficient in early test scenario prioritization and detection of faults.

K-Step Crossover Method based on Genetic Algorithm for Test Suite Prioritization in Regression Testing

Software is an integration of numerous programming modules  (e.g., functions, procedures, legacy system, reusable components, etc.) tested and combined to build the entire module. However, some undesired faults may occur due to a change in modules while performing validation and verification. Retesting of entire software is a costly affair in terms of money and time. Therefore, to avoid retesting of entire software, regression testing is performed. In regression testing, an earlier created test suite is used to retest the software system's modified module. Regression Testing works in three manners; minimizing test cases, selecting test cases, and prioritizing test cases. In this paper, a two-phase algorithm has been proposed that considers test case selection and test case prioritization technique for performing regression testing on several modules ranging from a smaller line of codes to huge line codes of procedural language. A textual based differencing algorithm has been implemented for test case selection. Program statements modified between two modules are used for textual differencing and utilized to identify test cases that affect modified program statements. In the next step, test case prioritization is implemented by applying the Genetic Algorithm for code/condition coverage. Genetic operators: Crossover and Mutation have been applied over the initial population (i.e. test cases), taking code/condition coverage as fitness criterion to provide a prioritized test suite. Prioritization algorithm can be applied over both original and reduced test suite depending upon the test suite's size or the need for accuracy. In the obtained results, the efficiency of the prioritization algorithms has been analyzed by the Average Percentage of Code Coverage (APCC) and Average Percentage of Code Coverage with cost (APCCc). A comparison of the proposed approach is also done with the previously proposed methods and it is observed that APCC & APCCc values achieve higher percentage values faster in the case of the prioritized test suite in contrast to the non-prioritized test suite.

Automation of Selection of a Pool of Graphical Interface Regression Tests for Multi Module Information Systems

Features of using the regression test selection method for automated testing of the graphical user interface in the development of information systems that consist of a set of modules are considered. The source of the need to create additional test environments required in the development of multi-module information systems that are using databases is specified. The three most popular approaches to organizing test environments – Copying, Scaling, and Scaling with synthetic data generation – are considered. The positive and negative sides are considered in terms of implementation, using, and resources spent on creating and maintaining resources, as well as in terms of the reliability of the results obtained in the process of testing models created using these approaches. The positive aspects of checking the quality of complex multi-module information systems from the point of view of the graphical user interface by various testing methods and, in particular, in the process of performing regression testing are presented. The positive aspects of using regression testing automation in conditions of lack of resources using various software platforms are indicated. The advantages of using the dynamic selection method for regression tests for automated testing are also given, as well as recommendations for implementing the selection method in existing and beginning projects.

Prioritizing versions for performance regression testing: The Pharo case

• An empirical study of real-world performance variations. • A sampling technique to statically identify performance regressions. • A prototype and an empirical evaluation. Software performance may suffer regressions caused by source code changes. Measuring performance at each new software version is useful for early detection of performance regressions. However, systematically running benchmarks is often impractical ( e.g., long running execution, prioritizing functional correctness over non-functional). In this article, we propose Horizontal Profiling , a sampling technique to predict when a new revision may cause a regression by analyzing the source code and using run-time information of a previous version. The goal of Horizontal Profiling is to reduce the performance testing overhead by benchmarking just software versions that contain costly source code changes. We present an evaluation in which we apply Horizontal Profiling to identify performance regressions of 17 software projects written in the Pharo programming language, totaling 1,288 software versions. Horizontal Profiling detects more than 80% of the regressions by benchmarking less than 20% of the versions. In addition, our experiments show that Horizontal Profiling has better precision and executes the benchmarks in less versions that the state of the art tools, under our benchmarks. We conclude that by adequately characterizing the run-time information of a previous version, it is possible to determine if a new version is likely to introduce a performance regression or not. As a consequence, a significant fraction of the performance regressions are identified by benchmarking only a small fraction of the software versions.

Benchmark Dataset Selection of Web Services Technologies: A Factor Analysis

Web services have emerged as an accessible technology with the standard ’Extensible Mark Up’ (XML) language, which is known as ’Web Services Description Language’ WSDL. Web services have become a promising technology to promote the interrelationship between service providers and users. Web services users’ trust is measured by quality metrics. Web service quality metrics vary in many benchmark datasets used in the existing studies. The selection of a benchmark dataset is problematic to classify and retest web services. This paper proposes a method to rank web services quality metrics for the selection of benchmark web services datasets. To measure the diversity in quality metrics, factor analysis with Varimax rotation and scree plot is a well-established method. We use factor analysis to determine percentage variance among principal factors of four benchmark datasets. Our results showed that the two-factor solution explained 94.501, 76.524, and 45.009% variances in datasets A, B, and D, respectively. A three-factor solution explained 85.085% variance in dataset C. Reliability, and response time quality metrics were predicted as the most dominating quality metrics that contributed to explain the percentage variance in four datasets. Our proposed web metric ranking (WMR) method resulted in reliability as the top-most web metric with (57.62%) score and latency web metric at the bottom-most with (3.60%) score. The proposed WMR method showed a high (96.17%) ranking precision. Obtained results verified that factor solutions after reducing the dimensions could be generalized and used in the quality improvement of web services. In future works, the authors plan to focus on a dataset with dominating quality metrics to perform regression testing of web services.

Adequate vs. inadequate test suite reduction approaches

Abstract Context: Regression testing is an important activity that allows ensuring the correct behavior of a system after changes. As the system grows, the time and resources to perform regression testing increase. Test Suite Reduction (TSR) approaches aim to speed up regression testing by removing obsolete or redundant test cases. These approaches can be classified as adequate or inadequate. Adequate TSR approaches reduce test suites and completely preserve test requirements (e.g., covered statements) of the original test suites. Inadequate TSR approaches do not preserve test requirements. The percentage of satisfied test requirements indicates the inadequacy level. Objective: We compare some state-of-the-art adequate and inadequate TSR approaches with respect to the size of reduced test suites and their fault-detection capability. We aim to increase our body of knowledge on TSR approaches by comparing: (i) well-known traditional adequate TSR approaches; (ii) their inadequate variants; and (iii) several variants of a novel Clustering-Based (CB) approach for (adequate and inadequate) TSR. Method: We conducted an experiment to compare adequate and inadequate TSR approaches. This comparison is founded on a public dataset containing information on real faults. Results: The most important findings from our experiment can be summarized as follows: (i) there is not an inadequate TSR approach that outperforms the others; (ii) some inadequate variants of the CB approach, and few traditional inadequate approaches, outperform the adequate ones in terms of reduction in test suite size with a negligible effect on fault-detection capability; and (iii) the CB approach is less sensitive than the other inadequate approaches, that is, variations in the inadequacy level have small effect on reduction in test suite size and on loss in fault-detection capability. Conclusions: These findings imply that inadequate TSR approaches and especially the CB approach might be appealing because they lead to a greater reduction in test suite size (with respect to the adequate ones) at the expense of a small loss in fault-detection capability.

SixTrack Version 5: Status and New Developments

SixTrack Version 5 is a major SixTrack release that introduces new features, with improved integration of the existing ones, and extensive code restructuring. New features include dynamic-memory management, scattering-routine integration, a new initial-condition module, and reviewed post-processing methods. Existing features like on-line aperture checking and Fluka-coupling are now enabled by default. Extensive performance regression tests have been developed and deployed as part of the new-release generation. The new features of the tracking environment developed for the massive numerical simulations will be discussed as well.

Study of influence of test case parameters on test suite optimality in order to automate the information systems design process

The paper presents data on the influence of test parameters like average execution time, number of runs, test’s severity and complexity, number of related bugs on test’s optimality to include a test set for regression testing of information system software components. The paper includes research using such methods as analysis of variance, correlation and regression analysis of test parameters. The studies have shown a degree of the influence of test’s parameters to test suite efficiency and optimality in order to perform regression testing. Solving the problem of creation of optimal test suites aimed at regression testing gives a chance to automate the information systems design process.

Automated code-based test selection for software product line regression testing

Regression testing for software product lines (SPLs) is challenging and can be expensive because it must ensure that all the products of a product family are correct whenever changes are made. SPL regression testing can be made efficient through a test case selection method that selects only the test cases relevant to the changes. Some approaches for SPL test case selection have been proposed but either they were not efficient by requiring intervention from human experts or they cannot be used if requirements specifications, architecture and/or traceabilities for test cases are not available or partially eroded. To address these limitations, we propose an automated method of source code-based regression test selection for SPLs. Our method reduces the repetition of the selection procedure and minimizes the in-depth analysis effort for source code and test cases based on the commonality and variability of a product family. Evaluation results of our method using six product lines show that our method reduces the overall time to perform regression testing by 14.8% ∼ 49.1% on average compared to an approach of repetitively applying Ekstazi, which is the state-of-the-art regression test selection method for a single product, to each product of a product family.

ICR142 Benchmarker: evaluating, optimising and benchmarking variant calling performance using the ICR142 NGS validation series.

Evaluating, optimising and benchmarking of next generation sequencing (NGS) variant calling performance are essential requirements for clinical, commercial and academic NGS pipelines. Such assessments should be performed in a consistent, transparent and reproducible fashion, using independently, orthogonally generated data. Here we present ICR142 Benchmarker, a tool to generate outputs for assessing germline base substitution and indel calling performance using the ICR142 NGS validation series, a dataset of Illumina platform-based exome sequence data from 142 samples together with Sanger sequence data at 704 sites.ICR142 Benchmarker provides summary and detailed information on the sensitivity, specificity and false detection rates of variant callers. ICR142 Benchmarker also automatically generates a single page report highlighting key performance metrics and how performance compares to widely-used open-source tools. We used ICR142 Benchmarker with VCF files outputted by GATK, OpEx and DeepVariant to create a benchmark for variant calling performance. This evaluation revealed pipeline-specific differences and shared challenges in variant calling, for example in detecting indels in short repeating sequence motifs. We next used ICR142 Benchmarker to perform regression testing with DeepVariant versions 0.5.2 and 0.6.1. This showed that v0.6.1 improves variant calling performance, but there was evidence of minor changes in indel calling behaviour that may benefit from attention. The data also allowed us to evaluate filters to optimise DeepVariant calling, and we recommend using 30 as the QUAL threshold for base substitution calls when using DeepVariant v0.6.1. Finally, we used ICR142 Benchmarker with VCF files from two commercial variant calling providers to facilitate optimisation of their in-house pipelines and to provide transparent benchmarking of their performance. ICR142 Benchmarker consistently and transparently analyses variant calling performance based on the ICR142 NGS validation series, using the standard VCF input and outputting informative metrics to enable user understanding of pipeline performance. ICR142 Benchmarker is freely available at https://github.com/RahmanTeamDevelopment/ICR142_Benchmarker/releases.

ICR142 Benchmarker: evaluating, optimising and benchmarking variant calling performance using the ICR142 NGS validation series

Evaluating, optimising and benchmarking of next generation sequencing (NGS) variant calling performance are essential requirements for clinical, commercial and academic NGS pipelines. Such assessments should be performed in a consistent, transparent and reproducible fashion, using independently, orthogonally generated data. Here we present ICR142 Benchmarker, a tool to generate outputs for assessing germline base substitution and indel calling performance using the ICR142 NGS validation series, a dataset of Illumina platform-based exome sequence data from 142 samples together with Sanger sequence data at 704 sites. ICR142 Benchmarker provides summary and detailed information on the sensitivity, specificity and false detection rates of variant callers. ICR142 Benchmarker also automatically generates a single page report highlighting key performance metrics and how performance compares to widely-used open-source tools. We used ICR142 Benchmarker with VCF files outputted by GATK, OpEx and DeepVariant to create a benchmark for variant calling performance. This evaluation revealed pipeline-specific differences and shared challenges in variant calling, for example in detecting indels in short repeating sequence motifs. We next used ICR142 Benchmarker to perform regression testing with DeepVariant versions 0.5.2 and 0.6.1. This showed that v0.6.1 improves variant calling performance, but there was evidence of minor changes in indel calling behaviour that may benefit from attention. The data also allowed us to evaluate filters to optimise DeepVariant calling, and we recommend using 30 as the QUAL threshold for base substitution calls when using DeepVariant v0.6.1. Finally, we used ICR142 Benchmarker with VCF files from two commercial variant calling providers to facilitate optimisation of their in-house pipelines and to provide transparent benchmarking of their performance. ICR142 Benchmarker consistently and transparently analyses variant calling performance based on the ICR142 NGS validation series, using the standard VCF input and outputting informative metrics to enable user understanding of pipeline performance. ICR142 Benchmarker is freely available at https://github.com/RahmanTeamDevelopment/ICR142_Benchmarker/releases.

ICR142 Benchmarker: evaluating, optimising and benchmarking variant calling using the ICR142 NGS validation series.

Evaluating, optimising and benchmarking of next generation sequencing (NGS) variant calling performance are essential requirements for clinical, commercial and academic NGS pipelines. Such assessments should be performed in a consistent, transparent and reproducible fashion, using independently, orthogonally generated data. Here we present ICR142 Benchmarker, a tool to generate outputs for assessing variant calling performance using the ICR142 NGS validation series, a dataset of exome sequence data from 142 samples together with Sanger sequence data at 704 sites. ICR142 Benchmarker provides summary and detailed information on the sensitivity, specificity and false detection rates of variant callers. ICR142 Benchmarker also automatically generates a single page report highlighting key performance metrics and how performance compares to widely-used open-source tools. We used ICR142 Benchmarker with VCF files outputted by GATK, OpEx and DeepVariant to create a benchmark for variant calling performance. This evaluation revealed pipeline-specific differences and shared challenges in variant calling, for example in detecting indels in short repeating sequence motifs. We next used ICR142 Benchmarker to perform regression testing with versions 0.5.2 and 0.6.1 of DeepVariant. This showed that v0.6.1 improves variant calling performance, but there was evidence of some minor changes in indel calling behaviour that may benefit from attention in future updates. The data also allowed us to evaluate filters to optimise DeepVariant calling, and we recommend using 30 as the QUAL threshold for base substitution calls when using DeepVariant v0.6.1. Finally, we used ICR142 Benchmarker with VCF files from two commercial variant calling providers to facilitate optimisation of their in-house pipelines and to provide transparent benchmarking of their performance. ICR142 Benchmarker consistently and transparently analyses variant calling performance based on the ICR142 NGS validation series, using the standard VCF input and outputting informative metrics to enable user understanding of pipeline performance. ICR142 Benchmarker is freely available at https://github.com/RahmanTeamDevelopment/ICR142_Benchmarker/releases.

High-performance, robustly verified Monte Carlo simulation with FullMonte.

We introduce the FullMonte tetrahedral 3-D Monte Carlo (MC) software package for simulation, visualization, and analysis of light propagation in heterogeneous turbid media including tissue. It provides the highest computational performance and richest set of input, output, and analysis facilities of any open-source tetrahedral-mesh MC light simulator. It also provides a robust framework for statistical verification. A scripting interface makes set-up of simulation runs simple, including parameter sweeps, while simultaneously providing customization options. Data formats shared with class-leading visualization tools, VTK and Paraview, facilitate interactive generation of publication-quality fluence and irradiance maps. The simulator can read and write file formats supported by other similar simulators, such as TIM-OS, MMC, COMSOL (finite-element simulations), and MCML to support comparison. Where simulator features permit, FullMonte can take a single test case, run it in multiple software packages, and load the results together for comparison. Example meshes, optical properties, set-up scripts, and output files are provided for user convenience. We demonstrate its use in several test cases, including photodynamic therapy of the brain, bioluminescence imaging (BLI) in a mouse phantom, and a comparison against MCML for layered geometries. Application domains that can benefit from use of FullMonte include photodynamic, photothermal, and photobiomodulation therapies, BLI, diffuse optical tomography, MC software development, and biophotonics education. Since MC results may be used for preclinical or even clinical experiments, a robust and rigorous verification process is essential. Being a stochastic numerical method, MC simulation has unique challenges associated with verification of output results since observed differences may be due simply to output variance or actual differences in expected output. We describe and have implemented a rigorous and statistically justified framework for comparing between simulators of the same class and for performing regression testing.

Supporting inheritance hierarchy changes in model-based regression test selection

Models can be used to ease and manage the development, evolution, and runtime adaptation of a software system. When models are adapted, the resulting models must be rigorously tested. Apart from adding new test cases, it is also important to perform regression testing to ensure that the evolution or adaptation did not break existing functionality. Since regression testing is performed with limited resources and under time constraints, regression test selection (RTS) techniques are needed to reduce the cost of regression testing. Applying model-level RTS for model-based evolution and adaptation is more convenient than using code-level RTS because the test selection process happens at the same level of abstraction as that of evolution and adaptation. In earlier work, we proposed a model-based RTS approach called MaRTS to be used with a fine-grained model-based adaptation framework that targets applications implemented in Java. MaRTS uses UML models consisting of class and activity diagrams. It classifies test cases as obsolete, reusable, or retestable based on changes made to UML class and activity diagrams of the system being adapted. However, MaRTS did not take into account the changes made to the inheritance hierarchy in the class diagram and the impact of these changes on the selection of test cases. This paper extends MaRTS to support such changes and demonstrates that the extended approach performs as well as or better than code-based RTS approaches in safely selecting regression test cases. While MaRTS can generally be used during any model-driven development or model-based evolution activity, we have developed it in the context of runtime adaptation. We evaluated the extended MaRTS on a set of applications and compared the results with code-based RTS approaches that also support changes to the inheritance hierarchy. The results showed that the extended MaRTS selected all the test cases relevant to the inheritance hierarchy changes and that the fault detection ability of the selected test cases was never lower than that of the baseline test cases. The extended MaRTS achieved comparable results to a graph-walk code-based RTS approach (DejaVu) and showed a higher reduction in the number of selected test cases when compared with a static analysis code-based RTS approach (ChEOPSJ).