Challenges of Specification Mining-Based Test Oracle for Cyber-Physical Systems

Abstract

Test oracle problem is considered as a major challenge in software testing. Specification mining techniques are shown to be effective to tackle test oracle problem in software systems. In addition, modern systems such as Cyber-Physical Systems (CPSes) have special constraints that should be satisfied when deriving the test oracles for these systems. However, comparing different specification mining techniques for CPS applications is challenging, because no common ground to assess the effectiveness of such techniques has been established yet. In this survey, our contribution is two folded: First, we analyze the CPS constraints from the test oracle point of view, and present a framework of requirements representing six essential criteria for evaluating to which extent specification miners satisfy CPS constraints. Secondly, we review the literature for the specification mining techniques, and use our framework to compare the effectiveness of various static and dynamic analysis-based specification mining techniques, and to discuss their respective advantages and disadvantages,