The public understanding of measurement inaccuracy

Abstract

Assessment agencies are increasingly facing pressure on two fronts; first, to increase transparency and openness and second, to improve public confidence. Yet, in relation to one of the central concepts of educational measurement—inherent error—many believe that increased public understanding is incompatible with public confidence: a general recognition of the true nature and extent of measurement inaccuracy would fatally undermine trust in the system. The present article is premised on a contrary proposal: not understanding measurement inaccuracy is a far greater threat than understanding it, since it will result in the system repeatedly being held to account for more than it can possibly deliver. As unrealistic expectations are unmet, so the system will appear to have failed; and this recurrent process will gradually erode public confidence. The article develops ethical and practical arguments in favour of educating the public about the inherent limitations of educational measurement. Primary amongst the ethical arguments is the proposal, from contemporary validity theory, that users who fail to understand measurement inaccuracy will be ill equipped to draw valid inferences from results.