Assessing the health of plants: simulation helps us understand observer disagreements

Abstract

A dataset on the health of plants, as judged by two raters, appears to show more disagreement about the relatively healthy plants than about the less healthy. The bivariate normal distribution is shown to be a poor description of the data, and a new bivariate distribution is developed that gives a good fit to the data. The chief features of the distribution are that it is a model with latent variables in common (true score plus error for each rater), and that the scatter of error is greater when the true score is high than when it is low. As the distribution is fitted to the data by simulation, an explicit expression for the joint distribution of the two observed scores is not required. The software used has ranking and recoding commands of one line each, so it is easy to ensure the fitted marginal distributions exactly match the data, and it is unnecessary to estimate parameters representing the boundaries between the grades of rating. The method of obtaining the distribution is very flexible. In one illustration of this, the relation between true score and the logarithm of the scatter of error is quadratic (not linear); in another, the true score and the errors have non-normal distributions. Copyright © 2000 John Wiley & Sons, Ltd.