On an Approach to Testing and Modeling Competence

Abstract

E. L. Thorndike contributed significantly to the field of educational and psychological testing as well as more broadly to psychological studies in education. This article follows in his testing legacy. I address the escalating demand, across societal sectors, to measure individual and group competencies. In formulating an approach to measuring competence, I draw on measurement research I have done over my career; the Thorndike lecture is to be as much autobiographical as substantive and/or methodological. I present an approach to defining, measuring, and statistically modeling competency measurements. The article unpacks Hartig et al.'s (2008) definition of competence as a complex ability construct closely related to real-life-situated performance. The intent is to make the construct, competence, amenable to measurement. Once unpacked, criteria for building competence measurements are set forth and exemplified by research from business, military, and education sectors. Generalizability theory, a statistical theory for modeling and evaluating the dependability of competence scores, is applied to several of these examples. The article then pulls together the threads into a general competency measurement model and concludes by noting its limitations.