Can racial/ethnic subgroup criterion‐to‐test standard deviation ratios account for conflicting differential validity and differential prediction evidence for cognitive ability tests?

Abstract

Differential validity and differential prediction analyses have come to conflicting conclusions regarding whether the relationship between cognitive ability tests and performance is the same across racial/ethnic subgroups. A prominent criticism of differential validity analyses is that they are confounded by subgroup differences in the ratio of criterion‐to‐test standard deviations (SDs). We investigated whether subgroup differences in these ratios can account for this conflicting evidence. Drawing on data from over 1 million participants, we find that subgroup differences in criterion‐to‐test SD ratios in general only account for a relatively small portion of subgroup differences in test‐criterion correlations.Practitioner points Cognitive ability tests exhibit differential validity for Asian, Black, Hispanic, and White subgroups. In most domains, subgroup differences in criterion‐to‐test SD ratios only account for a relatively small portion of these validity differences and thus do not explain why validities differ across subgroups. Still, because a portion of the already‐small validity differences are accounted for by SD ratios, the magnitude of any subgroup regression slope differences is small and is unlikely to result in underprediction of criterion performance for Asian, Black, and Hispanic subgroups.