Increasing Validity of Simulation Models Through Metamorphic Testing

Abstract

Simulation validation, demonstrating that a simulation accurately represents the system it studies, poses a particularly potent form of the oracle problem. A simulation must be validated for its results to be reliable, but often no oracle exists due to the nature of simulations. We propose taking the metamorphic testing (MT) approach from the realm of software verification and applying it for validating executable simulation models. By establishing pseudo-oracles based on metamorphic relations between parameters and behaviors within an executable model, we can create a methodical approach for validating simulation models. We propose an overall framework and guidelines to apply MT for simulation validation, with details for two prevalent simulation approaches: agent-based simulation models (ABM), and discrete-event simulation (DES) modeling. Through three case studies, we demonstrate the successful application of MT toward validating three different simulation models: an ABM of gossip propagation, an ABM of cancer, and a DES of airport check-in and security.