The observation range adjusted method: a novel approach to accounting for observation uncertainty in model evaluation

Abstract

Model evaluations are performed by comparing a modelled quantity with an observation of that quantity and any deviation from this observed quantity is considered an error. We know that all observing systems have uncertainties, and multiple observational products for the same quantity can provide equally plausible ‘truths’. Thus, model errors depend on the choice of observation used in the evaluation exercise. We propose a method that considers models to be indistinguishable from observations when they lie within the range of observations, and hence are not assigned any error. Errors are assigned when models are outside the observational range. Errors calculated in this way can be used within traditional statistics to calculate the Observation Range Adjusted (ORA) version of that statistic. The ORA statistics highlight the measurable errors of models, provide more robust model performance rankings, and identify areas of the model where further model development is likely to lead to consistent model improvements.