Model goodness-of-fit analysis using regression and related techniques

Abstract

Four related approaches for assessing model goodness-of-fit (GOF) are discussed in this paper: linear regression of observed versus predicted values, the sum of squared prediction errors, a reliability index summarizing predictions as being within a factor K s of observed values, and correlation-like measures of fit that normalize the sum of squared prediction error to be between zero and one. Relationships among the four measures are derived and alternative decompositions of the measures into components relating to bias, variance, and consistency are presented. The measures are extended to include lack-of-fit terms when multiple observations are available for each prediction, and except for the reliability index, extended to include the multivariate case of multiple prediction variables. Application of the GOF measures to model predictions and observed radon-222 concentrations (a univariate example) showed significant lack-of-fit for high concentrations. Application of the GOF measures to predicted and observed mean lengths and densities of winter flounder larvae (a multivariate example) showed predicted densities were good with most of the lack-of-fit attributed to ∼ 0.44 mm bias in predicted mean lengths. The role of GOF analysis in evaluating model performance is discussed.