Abstract
Adequately testing scientific software is essential to the quality of the software. However, it is a grand challenge due to the oracle problem. Metamorphic testing has shown its effectiveness for alleviating the problem. But the effectiveness of metamorphic testing is highly dependent on the quality of metamorphic relations that are developed for testing the software. In this paper, we propose a framework for iteratively developing metamorphic relations for adequately testing scientific software. The basic idea is to refine metamorphic relations that are loosely defined to those that can be verified with only limited number of cases so that the relations can be accurately tested. We explain the framework through testing a scientific software system that is used for modeling light scattering of particles. Based on domain knowledge and general guidelines, a group of metamorphic relations are first identified and tested. According to testing results, the metamorphic relations are refined step by step until each of them can be accurately tested. In particular, an invariant transform is applied to the metamorphic relations for significantly reducing the number of cases that can satisfy the relations. Finally the relations are further transformed with a carefully defined hash function to ensure each of the metamorphic relations can be automatically verified. The proposed approach truly solves the oracle problem and greatly improves the effectiveness of metamorphic testing. Its effectiveness is evaluated by mutation testing and demonstrated by new problems found in the software.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.