Chance success due to guessing and non-independence of true scores and error scores in multiple-choice tests: computer trials with prepared distributions.

Abstract

The effect of chance success due to guessing upon the variance of multiple-choice test scores was estimated from prepared distributions of large numbers of scores. Each score consisted of an assumed “true score” component and an “error score” component generated by a computer. A large negative correlation was found between true scores and error scores and a positive correlation between error scores and error scores. The equation showing reliability in terms of components of variance was derived under the more restrictive assumption that there is a correlation between true scores and error scores, and the result [Formula: see text] was obtained. The fact that reliability can be positive even though error variance and observed variance are equal was discussed.