An evaluation of diagnostic tests and their roles in validating forest biometric models

Abstract

Model validation is an important part of model development. It is performed to increase the credibility and gain sufficient confidence about a model. This paper evaluated the usefulness of 10 statistical tests, five parametric and five nonparametric, in validating forest biometric models. The five parametric tests are the paired t test, the Χ2 test, the separate t test, the simultaneous F test, and the novel test. The five nonparametric tests are the Brown-Mood test, the Kolmogorov–Smirnov test, the modified Kolmogorov–Smirnov test, the sign test, and the Wilcoxon signed-rank test. Nine benchmark data sets were selected to evaluate the behavior of these tests in model validation; three were collected from Alberta and six were published elsewhere. It was shown that the usefulness of statistical tests in model validation is very limited. None of the tests seems to be generic enough to work well across a wide range of models and data. Each model passed one or more tests, but not all of them. Because of this, caution should be exercised when choosing a statistical test or several tests together to try to validate a model. It is important to reduce and remove any potential personal bias in selecting a favorite test, which can influence the outcome of the results.