Reliability and External Validity of Personality Test Scores: The Role of Person and Item Error.

Abstract

This article explores the suitability of a proposed "Dual" model, in which both people and items are sources of measurement error, by assessing how the test scores are expected to behave in terms of marginal reliability and external validity when the model holds. Analytical derivations are produced for predicting: (a) the impact of person and item errors in the amount of marginal reliability and external validity, as well as the occurrence of "ceiling" effects; (b) the changes in test reliability across groups with different average amounts of person error, and (c) the phenomenon of differential predictability. Two empirical studies are also used both as an illustration and as a check of the predicted results. Results show that the model-based predictions agree with existing evidence as well as with basic principles in classical test theory. However, the additional inclusion of individuals as a source of error leads to new explanations and predictions. The proposal and results provide new sources of information in personality assessment as well as of evidence of model suitability. They also help to explain some disappointing recurrent results.