Follow-up Test Cases are Better Than Source Test Cases in Metamorphic Testing: A Preliminary Study

Abstract

Recently, metamorphic testing (MT) has been used to augment test datasets by inclusion of new (follow-up) test cases constructed from existing (source) test cases using metamorphic relations (MRs). It has been reported that the augmented test datasets usually have higher fault detection capabilities. It is natural to ask which contributes to the improvement of the fault detection capabilities. To investigate this issue, we conducted an empirical study on three DNN models by feeding 70,000 handwritten digits images, in which six sets of MRs were designed. We found that follow-up test cases have better fault detection capabilities than source test cases. Furthermore, the impacts of the amounts of follow-up test cases on the fault detection capabilities of the augmented test datasets were investigated.