Reducing adverse impact in high-stakes testing

Abstract

A critical goal for psychological science in the 21st century is to foster diversity, equity, and inclusion in occupational contexts. One arena which will continue to benefit from a focus on equity is high-stakes testing, such as the assessments used for personnel selection and classification decisions. We define an equitable test as one that minimizes group differences based on protected classes such as race, sex, and ethnicity, while predicting criterion performance equivalently across groups. In this article, we provide an overview of the concepts of test equity, adverse impact, and predictive bias. We discuss how group differences in performance on high-stakes tests such as the Armed Services Vocational Aptitude Battery (ASVAB) could be driven by differences in crystallized intelligence (i.e., acquired knowledge), which is emphasized by the ASVAB subtests and related to socioeconomic status. We suggest that shifting the focus of some high-stakes assessments away from crystallized intelligence or supplementing them with other cognitive constructs could mitigate group differences in performance without sacrificing criterion validity. In particular, we provide evidence that tests of attention control—the domain-general ability to maintain focus on task-relevant information and resist distraction—could provide a more equitable path forward.