Bridging the model-to-code abstraction gap with fuzzy logic in model-based regression test selection

Walter Cazzola,Mohammed Al-Refai,Sudipto Ghosh,Gabriele Maurina

doi:10.1007/s10270-021-00899-6

Abstract

Regression test selection (RTS) approaches reduce the cost of regression testing of evolving software systems. Existing RTS approaches based on UML models use behavioral diagrams or a combination of structural and behavioral diagrams. However, in practice, behavioral diagrams are incomplete or not used. In previous work, we proposed a fuzzy logic based RTS approach called FLiRTS that uses UML sequence and activity diagrams. In this work, we introduce FLiRTS 2, which drops the need for behavioral diagrams and relies on system models that only use UML class diagrams, which are the most widely used UML diagrams in practice. FLiRTS 2 addresses the unavailability of behavioral diagrams by classifying test cases using fuzzy logic after analyzing the information commonly provided in class diagrams. We evaluated FLiRTS 2 on UML class diagrams extracted from 3331 revisions of 13 open-source software systems, and compared the results with those of code-based dynamic (Ekstazi) and static (STARTS) RTS approaches. The average test suite reduction using FLiRTS 2 was 82.06%. The average safety violations of FLiRTS 2 with respect to Ekstazi and STARTS were 18.88% and 16.53%, respectively. FLiRTS 2 selected on average about 82% of the test cases that were selected by Ekstazi and STARTS. The average precision violations of FLiRTS 2 with respect to Ekstazi and STARTS were 13.27% and 9.01%, respectively. The average mutation score of the full test suites was 18.90%; the standard deviation of the reduced test suites from the average deviation of the mutation score for each subject was 1.78% for FLiRTS 2, 1.11% for Ekstazi, and 1.43% for STARTS. Our experiment demonstrated that the performance of FLiRTS 2 is close to the state-of-art tools for code-based RTS but requires less information and performs the selection in less time.

Highlights

Regression testing is an expensive activity that ensures that changes made to evolving software systems do not break previously tested functionality [32,61]
This is because FLiRTS 2 is designed to work with UML class diagrams and this diagram type cannot provide certain types of information used by the other tools such as (1) dynamic dependencies of test classes with the classes under test, (2) Java reflection, (3) exceptions, (4) dependencies from test classes to third party libraries and input configuration files, and (5) the code contained inside the method body
We investigated the use of fuzzy logic in regression test selection to bridge the abstraction gap between the source code and the UML class diagrams that model structural aspects of the system

Summary

Introduction

Regression testing is an expensive activity that ensures that changes made to evolving software systems do not break previously tested functionality [32,61]. RTS is performed by analyzing the changes made to software at the code [32] or model level [11]. Leung and White [47] show that a selective-retest technique becomes beneficial when the cost of test selection is less than the cost of running the non-selected tests. This makes it evident that reducing the cost of the selection process is important. Briand et al [11] state that early estimation of regression testing effort is possible after modifying the design of the system and before propagating the changes to the code [11]

Methods

Results

Conclusion