GRE predictive validity in a Master's program in clinical psychology.

Abstract

The common use of Graduate Record Examination (GRE) scores to inform graduate program selection decisions has continued despite reservations that scores might not predict much more than graduate-level course grades. For example, Sternberg and Williams (1997) have investigated the predictive validity of GRE scores at Yale University, and found that scores predicted around 4% of the variance of first-year grades. However, when they related GRE scores to internal and external ratings of the quality of dissertation produced by students, only the GRE Analytical score was predictive, and only for male students. In contrast, the GRE Technical Manual (Briel, O'Neill, & Scheuneman, 1993) reports a number of studies in which GRE scores modestly predict both first-year grades as well as faculty ratings of performance, although these relations are not moderated by gender. These weak patterns justifiably give pause to many faculty involved with graduate programs that require GRE scores (as in Ceci, 1996), given the critical nature of admission decisions for applicants.High-quality, well-known graduate programs invariably have students with very high GRE scores. Sternberg and Williams (1997), for example, report means that range from 648 to 699 on subtests. Studies often have an adequate range in scores, but the distributions centre around the 90th percentile, few students have relatively low scores (e.g., below 50th percentiles), and students with no scores are either not included in analyses or not admitted to programs in the first instance (Darlington, 1998; Kuncel, Campbell, & Ones, 1998; Ruscio, 1998). Analyses are often based on correlational statistics, but predictive relations can also be defined by using cut-off scores that discriminate more from less successful students. This approach is only possible, however, when sufficient proportions of students score below cutoffs. In addition, outcomes are potentially biased and unreliable (Kuncel et al., 1998; Thayer & Kalat, 1998), particularly when they are embedded within the program itself (e.g., grades, faculty ratings). Reliable and objective external criteria would seem critical to use as outcome measures, although the selection of these may differ from program to program depending how success is defined for graduates.Whereas predictive validity data almost exclusively come from Ph.D. programs in the United States, some Master's programs also use GRE scores in admissions procedures. In Canada, Master's programs vary between those: (a) within a Ph.D. program where most or some students continue on; (b) that do not require GRE scores; (c) that are geared almost exclusively towards producing practitioners; and, (d) that produce students that either continue on in graduate studies elsewhere or seek employment. Predictive validity studies would need to differ quite dramatically between these different kinds of programs because the presence of predictor and nature of criterion variables would vary. The purpose of the current report is to present predictive validity data collected in our Master's program in clinical psychology at Acadia University, and thus what follows can be considered a case study. Ours is one of few programs in Canada that fits the fourth kind of program, but this offers methodological advantages when choosing important and external criterion variables.Our program typically admits 4 or 5 students per year, and the current sample consists of 37 students (29 women, 8 men) who have completed their 2-year residency requirement in the program over the past 10 years. Upon application, 12 of the admitted students did not provide GRE scores, and of the other 25, 4 did not write the Advanced test in psychology. Cumulative undergraduate 4-point GPAs and 4 GRE scores were used as predictors, and these means (see Table 1) are lower than would be found in Ph.D. graduate programs in larger schools or Ph.D.-level programs (see listing in APA, 1996). …