Subgroup Mean Differences Research Articles

Stereotype lift and stereotype threat effects on subgroup mean differences for cognitive tests: A meta-analysis of adult samples.

A large body of literature has studied the effect of stereotype threat and stereotype lift on cognitive test performance. Research on stereotype threat (ST) examines whether the awareness of a negative stereotype can decrease stereotyped group members' test performance. A less commonly studied influence of stereotypes is stereotype lift (SL), defined as an increase in a group's test performance due to not being part of a negative stereotype. For example, men might perform better on math tests if they are primed on the stereotype that men are better than women at math. Walton and Cohen (2003) previously meta-analyzed the impact of SL on cognitive tests, finding an overall d = 0.24. We report an updated meta-analysis on SL with more samples and moderator analyses. We then meta-analyzed between-group effects (majority-minority group differences both in the presence and absence of SL and ST) to compare their relative contributions to subgroup mean differences on cognitive tests. Our results indicate that SL has a small influence on cognitive test performance (d = 0.09, SDres = 0.19), and that subgroup mean differences result largely from between-group effects rather than from the effects of ST and SL. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

The relationship between cognitive-ability saturation and subgroup mean differences across predictors of job performance.

The authors quantify the conventional wisdom that predictors' correlations with cognitive ability are positively related to subgroup mean differences. Using meta-analytic and large-N data from diverse predictors, they found that cognitive saturation correlates .84 with predictors' artifact-corrected Black-White d values and .95 with predictors' artifact-corrected Hispanic-White d values. The authors also investigate the extent to which d values are associated with the use of assessor-based scoring and with predictor domains in which differential investment is likely to occur. As a practical application of these findings, they present a procedure to forecast mean differences on a new predictor based on its cognitive saturation and other attributes. They also present a Bayesian framework that allows one to integrate regression-based forecasts with observed d values to achieve more precise estimates of mean differences. The proposed forecasting techniques based on the relationship between mean differences and cognitive saturation can help to mitigate the difficulties inherent in computing precise local estimates of mean differences. (PsycINFO Database Record

Supplemental Material for The Relationship Between Cognitive-Ability Saturation and Subgroup Mean Differences Across Predictors of Job Performance

Supplemental Material for The Relationship Between Cognitive-Ability Saturation and Subgroup Mean Differences Across Predictors of Job Performance

Within‐High‐School Versus Across‐High‐School Scaling of Admissions Assessments: Implications for Validity and Diversity Effects

Previous research has established that SAT scores and high school grade point average (HSGPA) differ in their predictive power and in the size of mean differences across racial/ethnic groups. However, the SAT is scaled nationally across all test takers while HSGPA is scaled locally within a school. In this study, the researchers propose that this difference in how SAT scores and HSGPA are scaled partially explains differences in validity and subgroup differences. Using a large data set consisting of 170,390 students each of whom matriculated at one of 114 separate colleges, the researchers find that awarding SAT scores by ranking SAT within a high school generally results in substantial reduction in the size of subgroup mean differences for this predictor. However, validity for predicting first‐year GPA is also reduced by a small amount. Conversely, placing HSGPA onto a nationally normed metric through the use of multiple regression procedures results in a moderate increase in the size of subgroup mean differences, while also producing a small increase in validity. Taken together, these findings suggest that differences in predictor scaling can partially explain differences in the size of subgroup mean differences between HSGPA and SAT scores and have implications for predictive power.

Differential prediction generalization in college admissions testing.

We introduce the concept of differential prediction generalization in the context of college admissions testing. Specifically, we assess the extent to which predicted first-year college grade point average (GPA) based on high-school grade point average (HSGPA) and SAT scores depends on a student’s ethnicity and gender and whether this difference varies across samples. We compared 257,336 female and 220,433 male students across 339 samples, 29,734 Black and 304,372 White students across 247 samples, and 35,681 Hispanic and 308,818 White students across 264 samples collected from 176 colleges and universities between the years 2006 and 2008. Overall, results show a lack of differential prediction generalization because variability remains after accounting for methodological and statistical artifacts including sample size, range restriction, proportion of students across ethnicity- and gender-based subgroups, subgroup mean differences on the predictors (i.e., HSGPA, SAT-Critical Reading, SAT-Math, and SAT-Writing), and SDs for the predictors. We offer an agenda for future research aimed at understanding several contextual reasons for a lack of differential prediction generalization based on ethnicity and gender. Results from such research will likely lead to a better understanding of the reasons for differential prediction and interventions aimed at reducing or eliminating it when it exists. (PsycINFO Database Record (c) 2016 APA, all rights reserved)

Combining predictors to achieve optimal trade-offs between selection quality and adverse impact.

The authors propose a procedure to determine (a) predictor composites that result in a Pareto-optimal trade-off between the often competing goals in personnel selection of quality and adverse impact and (b) the relative importance of the quality and impact objectives that correspond to each of these trade-offs. They also investigated whether the obtained Pareto-optimal composites continue to perform well under variability of the selection parameters that characterize the intended selection decision. The results of this investigation indicate that this is indeed the case. The authors suggest that the procedure be used as one of a number of potential strategies for addressing the quality-adverse impact problem in settings where estimates of the selection parameters (e.g., validity estimates, predictor intercorrelations, subgroup mean differences on the predictors and criteria) are available from either a local validation study or meta-analytic research.

Impact of elaboration on socially desirable responding and the validity of biodata measures.

The current study investigated the impact of requiring respondents to elaborate on their answers to a biodata measure on mean scores, the validity of the biodata item composites, subgroup mean differences, and correlations with social desirability. Results of this study indicate that elaborated responses result in scores that are much lower than nonelaborated responses to the same items by an independent sample. Despite the lower mean score on elaborated items, it does not appear that elaboration affects the size of the correlation between social desirability and responses to biodata items or that it affects criterion-related validity or subgroup mean differences in a practically significant way.

The Relevance of the New APA Standards for Educational and Psychological Testing for Employment Testing in South Africa

Some of the major developments, reflected in the most recent revision of the Standards for educational and psychological testing of the American Psychological Association and two other sponsoring organizations, are reviewed. In view of the South African Employment Equity Act of 1998, the focus is on psychometric considerations that have a bearing on employment testing. Topics covered include the, conceptualisation of construct validity as the primary objective in test validation; the introduction of several sources of (construct) validity evidence; the formulation of requirements such as freedom from predictive bias, fairness and consideration of the consequences of testing; approaches to preventing unnecessary subgroup mean differences; and the appropriate use of non-test information when relevant norms are lacking. Finally, the implications of these psychometric developments for the training of local assessment practitioners are pointed out.

Traditional tests and job simulations: minority and majority performance and test validities.

Performance of 565 minority and 414 majority job applicants on traditional paper-and-pencil tests and on a telephone simulation that measured similar constructs was assessed. Models of the measurement characteristics of these 2 test batteries indicated larger subgroup mean differences on the traditional tests than on the simulation. Correlations between traditional tests were lower in the majority sample than in the minority sample, and the variance of the minority candidates' scores on the traditional tests was much larger than the variance of majority applicants' scores on the same measures. The validity of the simulation was lower than the validity of the traditional tests. This study replicates previous laboratory research that has indicated smaller subgroup differences on simulations than on paper-and-pencil tests and extends this research by providing evidence of the relative validity of these 2 types of measures.

Full-Time and Part-Time Subgroup Differences in Job Attitudes and Demographic Characteristics

This study examined the hypothesis that subgroups of part-timers differ from each other and from full-timers in their demographic characteristics, organizational commitment, and other job-related perceptions. Frame of reference differences and differing patterns of relationship investments are thought to account for these differences. We examined these group differences using samples of 850 full-time and 1490 part-time unionized employees. The part-time subgroups included moonlighters (n = 142), college students (n = 365), people providing supplemental income to their family (n = 661), and people whose part-time job was the primary source of income for their family (n = 556). The results indicated other demographic differences in these subgroups and indicated subgroup mean differences in several job attitudes and perceptions. Further, results of a regression analysis indicated some group differences in prediction of organizational commitment. These findings suggest that researchers must rethink the relations between work status and organizational attitudes and behavior. Research and practice implications are discussed.

PREDICTION OF TRADES APPRENTICES’PERFORMANCE ON JOB SAMPLE CRITERIA

A criterion‐related validation was conducted to assess the validity of four aptitude tests and five tests of content taken directly from job tasks in predicting job sample performance of apprentices in eight skilled trades. Observed validities were above .40 (corrected for range restriction, validities averaged .52). Though there were large subgroup mean differences on both predictor and criterion measures, there was no evidence of significant differential prediction.

Behavior of the Product-Moment Correlation Coefficient When Two Heterogeneous Subgroups Are Pooled1

An equation was derived to determine the relationship between the pooled within-subgroup correlation coefficient, rw, and the correlation coefficient obtained from the total group data, rt. It was, thus, possible to assess the amount of distortion introduced by pooling heterogeneous subgroups. As a basis for deciding whether to pool two subgroups in order to calculate rt, a two-stage procedure was recommended: (1) comparison of the two within-subgroup r's; and (2) comparison of rt and rw. On the basis of results for the second stage test, distortion in rt was shown to be a function of the pattern of subgroup mean differences, total group sample size, and the magnitude of rw. Implications were discussed.