THE RELATIONSHIP OF SCORES BASED ON GRE GENERAL TEST ITEM TYPES TO UNDERGRADUATE GRADES: AN EXPLORATORY STUDY FOR SELECTED SUBGROUPS

Abstract

ABSTRACTThis study used multiple discriminant analysis (MDA) to assess differences among subgroups defined in terms of sex and undergraduate major area, and ethnic group and undergraduate major area, with respect to patterns of performance on GRE item‐type part scores. Special subscores based on item‐types included in the current GRE General Test were derived for the study. The correlations of departmentally standardized scores on these subtests with a similarly standardized self‐reported undergraduate GPA (SR‐UGPA) criterion were analyzed. Multiple regression analysis was used to determine the relative contribution of the item‐type part scores to prediction for the various subgroups. Predicted SR‐UGPA means for subgroups, based on general major‐area regression equations using GRE item‐type part scores as predictors were compared with comparable predicted means using GRE total scores as predictors.The study was based on data from GRE files for 9,375 examinees in 12 fields of study, representing 437 undergraduate departments from 149 colleges and universities. Students were classified by field of study into four major areas: primarily verbal fields (English, history, sociology, political science); primarily quantitative fields (chemistry, computer science, mathematics, engineering, and economics); fields of mixed quantitative/verbal emphasis (Q/V), namely, biology and agriculture; and education.The GRE scores involved were (a) raw number‐right total verbal, quantitative, and analytical ability scores, unequated across test forms, transformed to a common scale–that is, z‐scaled by test form–labelled V*, Q*, and A*, respectively, to distinguish them from the corresponding GRE scaled scores; and (b) similarly developed verbal, quantitative, and analytical ability item‐type part scores. Verbal part scores were based on antonyms, analogies, sentence completion, and reading passage sets. Primary interest was in a vocabulary score (antonyms plus analogies) and a reading comprehension score (sentence completion plus reading passages). Quantitative part scores were based on quantitative comparison, regular mathematics, and data interpretation item types. Analytical ability part scores were based on analytical reasoning and logical reasoning item types.The part scores for each test were treated as individual variables in multiple discriminant analyses (MDA) for students classified by sex and major area, and by ethnic‐group membership and major area. For the MDA, the ethnic groups were American Indian, Black, Mexican American and other Hispanic, Puerto Rican, and Asian American; for regression analyses, by major area, the groups were Black, all Hispanic origin, Asian American, all Minority, White, male, and female.For each test, the criterion groups were found to be differentiated significantly along both a general ability dimension (represented by the principal discriminant function of part scores, all positively weighted), and a secondary, bipolar dimension (defined by a second significant discriminant function, uncorrelated with the general ability dimension, that reflected differences in patterns of performance on the part scores).The part scores (especially vocabulary and reading comprehension, and analytical reasoning and logical reasoning) were found to exhibit different patterns of correlations with the SR‐UGPA criterion.With respect to both patterns of part‐score means (mean scores on the second discriminant function) and patterns of part‐score correlations with SR‐UGPA, major‐area differences appeared to be stronger and more systematic than ethnic‐group or sex differences. Major‐area differences were more pronounced when verbal and analytical part scores were used as independent variables than when quantitative part scores were used. Systematic major‐area differences in patterns of part‐score/SR‐UGPA correlation were more clearly evident for reading comprehension and vocabulary part scores, and for analytical reasoning and logical reasoning part scores, than for the quantitative ability part scores.Using part scores rather than total scores did not result in different inferences regarding the relative standing of subgroups on the SR‐UGPA criterion. Predicted subgroup SR‐UGPA means based on general major‐area regression equations were essentially the same when item‐type part scores were used as predictors as when the three GRE section scores (V*, Q*, and A*) were used as predictors.Study findings indicate that the item‐type scores, especially scores based on verbal and analytical ability item types, provide more information about group differences than is provided by the total ability scores. Questions regarding the incremental predictive value of this information remain unresolved on the basis of the study findings. Sample size was limited for several of the subgroups, predictive equations were not cross‐validated, and self‐reported undergraduate grades rather than graduate grades were used as criteria. Resolution of these questions is a matter for further research. To be most useful, such research would involve graduate‐level performance criteria and employ equated part scores.Based on the overall pattern of findings, attention might most profitably be focused on the potential contribution of separate subscores for reading comprehension, vocabulary, analytical reasoning, and logical reasoning. Continued exploration of questions regarding the validity of item‐type part scores should contribute to better understanding of the nature of the abilities being measured by the GRE General Test, within if not beyond the well established verbal and quantitative domains.