Test oracles based on metamorphic relations for image processing applications

Abstract

Testing of image processing applications is a challenging job especially, when evaluating the correctness of output image. Generally, output images are evaluated manually by visual inspection carried out by an expert tester, which is the main hindrance in automation of testing process. Recently, statistical and metamorphic testing approaches are presented to automate output evaluation of image processing applications. The statistical method is dependent on availability of statistical distribution of output images, whereas metamorphic testing require more research efforts to make it widely used in practice. Metamorphic testing is a well-known technique to alleviate the test oracle problem and eliminates the required manual efforts by using relations of input and output images. Follow-up test cases are generated based on these relations and their expected output is evaluated. This paper addresses test oracle problem for image processing applications and demonstrates how properties of implementation under test can be adopted as metamorphic relations. We have studied general and specific metamorphic relations of morphological image operations such as dilation and erosion. Selection of metamorphic relations and their effectiveness by mutation analysis is demonstrated. The results show that metamorphic testing is useful for evaluation of output images in the absence of a perfect test oracle.