Science and Art of Setting Performance Standards and Cutoff Scores in Kinesiology

Abstract

Setting standards and cutoff scores is essential to any measurement and evaluation practice. Two evaluation frameworks, norm-referenced (NR) and criterion-referenced (CR), have often been used for setting standards. Although setting fitness standards based on the NR evaluation is relatively easy as long as a nationally representative sample can be obtained and regularly updated, it has several limitations—namely, time dependency, population dependence, discouraging low-level performers, and favoring advantaged or punishing disadvantaged individuals. Fortunately, these limitations can be significantly eliminated by employing the CR evaluation, which was introduced to kinesiology by Safrit and colleagues in the 1980s and has been successfully applied to some practical problems (e.g., set health-related fitness standards for FITNESSGRAM®). Yet, the CR evaluation has its own challenges, e.g., selecting an appropriate measure for a criterion behavior, when the expected relationship between the criterion behavior and a predictive measure is not clear, and when standards are not consistent among multiple field measures. Some of these challenges can be addressed by employing the latest statistical methods (e.g., test equating). This article provides a comprehensive review of the science and art of setting standards and cutoff scores in kinesiology. After a brief historical overview of the standard-setting practice in kinesiology is presented, a case analysis of a successful CR evaluation, along with related challenges, is described. Lessons learned from past and current practice as well as how to develop a defendable standard are described. Finally, future research needs and directions are outlined.