A Survey on Adaptive Random Testing

Rubing Huang,Xin Xia,Haibo Chen,Yinyin Xu,Weifeng Sun,Dave Towey

doi:10.1109/tse.2019.2942921

Abstract

Random testing (RT) is a well-studied testing method that has been widely applied to the testing of many applications, including embedded software systems, SQL database systems, and Android applications. Adaptive random testing (ART) aims to enhance RT's failure-detection ability by more evenly spreading the test cases over the input domain. Since its introduction in 2001, there have been many contributions to the development of ART, including various approaches, implementations, assessment and evaluation methods, and applications. This paper provides a comprehensive survey on ART, classifying techniques, summarizing application areas, and analyzing experimental evaluations. This paper also addresses some misconceptions about ART, and identifies open research challenges to be further investigated in the future work.

Highlights

S OFTWARE testing is a popular technique used to assess and assure the quality of the system under test (SUT)
The F-measure is the expected F-count [206]; the E-measure refers to the expected number of failures to be identified by a set of test cases; and the P-measure is the probability of a test set identifying at least one program failure
In addition to tracing the evolution and distribution of Adaptive random testing (ART) topics, we have classified the various ART approaches into different categories, analyzing their strengths and weaknesses

Summary

Introduction

S OFTWARE testing is a popular technique used to assess and assure the quality of the (software) system under test (SUT). One fundamental testing approach involves constructing test cases in a random manner from the input domain (the set of all possible program inputs): This approach is called random testing (RT) [1]. Two fundamental features can be used to describe the properties of the fault(s): the failure rate (the number of failure-causing inputs as a proportion of all possible inputs); and the failure pattern (the distributions of failure-causing inputs across the input domain, including their geometric shapes and locations). Before testing, these two features are fixed, but unknown

Methods

Results

Conclusion