Subgroup Differences in Situational Judgment Test Performance: A Meta-Analysis

Abstract

In this article, we describe a systematic review of mean race and sex differences in situational judgment test (SJT) performance. On average, White test takers perform better on SJTs than Black, Hispanic, and Asian test takers. Female examinees perform slightly better than male test takers on SJTs. We investigate two moderators of these differences: loading of g or personality on the SJT, and response instructions. Mean race differences between Black, Hispanic, Asian and White examinees in SJT performance are largely explained by the cognitive loading of the SJT such that the larger the cognitive load, the larger the mean race differences. Regarding the effect of personality loadings of SJTs on race differences, Black–White and Asian–White differences are smaller to the extent that the SJT is correlated with emotional stability and Hispanic–White differences are smaller to the extent that SJTs are correlated with conscientiousness and agreeableness. Cognitive loading has minimal effect on male–female SJT score differences; however, SJT score differences are larger, favoring women, when SJTs are correlated with conscientiousness and agreeableness. Concerning response instructions, knowledge response instructions showed greater race differences than behavioral tendency instructions. The mean correlations show that these differences are largely because of the greater g loading of knowledge instructions. A second study showed that when used in hiring, SJTs are likely to have adverse impact by race but not by sex.