Theory Testing and Measurement Error

Abstract

Accurate empirical tests of theories and hypotheses are not possible unless the inevitable biases induced into data by measurement error are controlled for. Yet despite 90 years of recommendations from measurement theory and methodology, some still do not control for these biases in their research. This paper presents simple and direct demonstrations showing why basic measurement principles require that biases in data created by measurement error be removed and refutes commonly heard objections to the corrections for these biases. One factor contributing to resistance on the part of some researchers is the fact that most psychologists are not aware that measurement error is produced by real psychological processes that can be studied and understood. This paper describes those substantive psychological process and shows how each generates a different type of measurement error. We also show how different types of reliability estimates assess and calibrate different error processes and types of measurement error, leading directly to conclusions about which types of reliability estimates are appropriate for measurement error corrections in different research settings. Failure to control for biases induced by measurement error has retarded the development of cumulative research knowledge. It is our hope that this paper will contribute to removing these hobbles from psychological research.