Confidence limits for air quality model evaluations, as estimated by bootstrap and jackknife resampling methods

Abstract

Listen

Translate article

Air quality models are used to make decisions regarding the construction of industrial plants, the types of fuel that will be burnt and the types of pollution control devices that will be used. It is important to know the uncertainties that are associated with these model predictions. Standard analytical methods found in elementary statistics textbooks for estimating uncertainties are generally not applicable since the distributions of performance measures related to air quality concentrations are not easily transformed to a Gaussian shape. This paper suggests several possible resampling procedures that can be used to calculate uncertainties or confidence limits on air quality model performance. In these resampling methods, many new data sets are drawn from the original data set using an empirical set of rules. A few alternate forms of the socalled bootstrap and jackknife resampling procedures are tested using a concocted data set with a Gaussian parent distributions, with the result that the jackknife is the most efficient procedure to apply, although its confidence bounds are slightly overestimated. The resampling procedures are then applied to predictions by seven air quality models for the Carpinteria coastal dispersion experiment. Confidence intervals on the fractional mean bias and the normalized mean square error are calculated for each model and for differences between models. It is concluded that these uncertainties are sometimes so large for data sets consisting of about 20 elements that it cannot be stated with 95% confidence that the performance measure for the ‘best’ model is significantly different from that for another model.