Statistical assessment of model goodness-of-fit using permutation tests ,

Abstract

Assessing model goodness-of-fit (GOF) to observed data is critical for establishing model credibility and confidence. We propose a new GOF approach based on permutation statistics. The approach is described for a simple, hypothetical example and demonstrated by comparing predictions of winter flounder length distributions from a population model to observed data from the Niantic River. Analyses of three different GOF measures, designed to evaluate the mean, S.D., and distribution of winter flounder lengths, showed that predicted mean lengths were indistinguishable from observed values while predicted S.D. and distributions differed significantly. Two other commonly used GOF approaches (regression and the Reynolds et al. (1981)approach of combining separate statistical tests) were also applied to the winter flounder example to illustrate the their relationship to permutation tests. The advantages of the permutation approach, and possible extensions to handle multiple predicted values, are discussed.