PM and light extinction model performance metrics, goals, and criteria for three-dimensional air quality models

Abstract

In order to use an air quality modeling system with confidence, model performance must be evaluated against observations. While ozone modeling and evaluation is fairly developed, particulate matter (PM) modeling is still an evolving science. EPA has issued minimal guidance on PM and visibility model performance evaluation metrics, goals, and criteria. This paper addresses these issues by examining various bias and error metrics and proposes PM model performance goals (the level of accuracy that is considered to be close to the best a model can be expected to achieve) and criteria (the level of accuracy that is considered to be acceptable for modeling applications) that vary as a function of concentration and extinction. In this paper, it has been proposed that a model performance goal has been met when both the mean fractional error (MFE) and the mean fractional bias (MFB) are less than or equal to +50% and ±30%, respectively. Additionally, the model performance criteria has been met when both the MFE⩽+75% and MFB⩽±60%. Less abundant species would have less stringent performance goals and criteria. These recommendations are based upon an analysis of numerous PM and visibility modeling studies performed throughout the country.