A Machine Learning Approach for Developing Test Oracles for Testing Scientific Software

Abstract

Absence of test oracles is the grand challenge for testing complex scientific software. Metamorphic testing is the novel technique for developing test oracles on metamorphic relations. Although it is easy to find metamorphic relations based on general guidelines and domain knowledge, the ones that can adequately test the software are difficult to be developed. This paper introduces a machine learning approach for iteratively developing metamorphic relations. The approach develops initial metamorphic relations and tests first, and then the relations and tests are refined through mining the initial test execution and evaluation results with machine learning algorithms. The approach and its effectiveness are illustrated through testing an open source discrete dipole approximation program. Keywords-metamorphic testing, metamorphic relation, test oracle, scientific software, machine learning.