SEARCH FOR PATTERNS OF SUCCESS ON GRE GENERAL TEST ITEMS

Abstract

ABSTRACTThe impetus for the present study comes from the rather inconclusive results from “test bias” studies, which can be characterized as studies to determine whether ethnic, racial, or gender groups differ from the rest of the population taking the GRE General Test in terms of patterns of item difficulty. The results of such studies have been sufficiently weak as to suggest that test items are not strongly “biased” for any particular subgroups of test takers as defined by race, gender, or ethnicity. It is possible, however, that (a) there are subgroups of test takers whose patterns of item difficulty differ but whose members are demographically mixed, or (b) there are no groups for whom item difficulty differences exist (except for overall level of performance on the test. The potential groupings that underlie test performance are called latent groups because they are not defined beforehand, but are a product of the study. Latent groups are defined as groups that differ in the patterns of their item difficulties.To clarify the results of bias studies, the present study sought to discover latent groups of examinees by utilizing only test performance data. Factor analyses were performed to determine lower bounds for the numbers of latent groups. Two types of coefficients were analyzed: the proportion of joint successes on pairs of items, and phi‐coefficients. Both types of coefficients were well accounted for by a single factor. This result implies the existence of no more than one group.To be sure that the effects of group heterogeneity were not being masked, test takers who described themselves as White were sampled down so that the responses of other demographic groups could more strongly effect the results. However, using either of the coefficients analyzed, a single factor still accounted very well for the relationships among test items.Clearly, if any latent group structure were to be detected, a more sensitive approach was required. Thus, the subsequent approach adopted examined the value of each residual after a given number of factors was extracted. This approach led to a three‐group solution for the verbal subtest, and a two‐group solution for the quantitative subtest. This pattern of item difficulties suggested that these groups were differentiated due to bends in the item characteristic curves, rather than through demographic effects. In addition, a large number of groups were identified for the analytic subtest, but the vector of item difficulties proved to be essentially proportional. Also, demographic and item content information proved uninterpretable. It was concluded that the tests were essentially unitary as intended.