ESTIMATING THE RELATIONSHIP BETWEEN USE OF TEST‐PREPARATION METHODS AND SCORES ON THE GRADUATE MANAGEMENT ADMISSION TEST

Abstract

ABSTRACTThis study sought to examine the relationship between five methods of test preparation and test performance as measured by Graduate Management Admission Test (GMAT) Verbal (V), Quantitative (Q) and Total (T) scores. Data on method of test preparation were obtained through voluntary examinee response to the following five questions which appeared on the answer sheets:In preparing for this test, did you: Study the sample questions in the GMAT registration bulletin? Work through an actual GMAT published by ETS? Use a book not published by ETS on how to prepare for the GMAT? Attend a test preparation or coaching course for the GMAT? Undertake on your own any review of mathematics? One sample of first‐time test takers and one sample of second‐time test takers were selected from among the 185,525 1981–82 GMAT examinees who were U.S. citizens. Multiple regressions using GMAT scores as dependent variables and test preparation, undergraduate grade point average (UGPA) and sex as independent variables were computed separately for first‐time examinees who were members of the Afro‐American/Black, Caucasion/White, Oriental/Asian and Spanish‐American U.S. citizen subgroups. Regressions (including first GMAT scores as independent variables) were also computed for all examinees in the sample who were taking the GMAT for the second time.FIRST‐TIME EXAMINEESThe percents of first‐time examinees electing to use each method varied, but the rank order of the frequency of using each method was consistent across the subgroups. The largest proportion of first‐time examinees reported that they had prepared by reviewing the bulletin, followed in descending order by using a test preparation book not prepared by ETS, undertaking their own study of mathematics, working through an actual GMAT, and attending a test preparation course. The study also found that examinees electing to use the various methods of preparation did not differ appreciably in previous academic performance as measured by undergraduate grade point average, but did vary slightly in age and amount of work experience.Results of the multiple regression analyses based on data from first‐time examinees differed across the four subgroups. The size of the coefficients associated with each method of preparation, as well as the corresponding standard errors, varied among the four subgroups. The same was true for the interaction effects between pairs of methods. The expected difference in verbal score for a “yes” response to “Studying a test review book not published by ETS” when the effects of the other independent variables were held constant, ranged from 1.6 to 3.2 scaled score points and were significant for all four subgroups. The difference in verbal scores for “Studying the Bulletin” ranged from 1.3 scaled score points for Afro‐American/Blacks to 4.0 scaled points for Oriental/Asians. The effects of using a review book or taking a review course ranged between .4 and 1.9 points on verbal and quantitative scores. Negative effects were associated with examinees' own review of mathematics. These effects were attributed to a confounding between self‐selection and method of preparation.SECOND‐TIME EXAMINEESThe sizes of the effects associated with each method of preparation for second‐time examinees were considerably less than those obtained using data from first‐time examinees. When previous test performance was held constant, the effect of using each of the methods of preparation was small. In fact, only the effects of using a test preparation book and of attending a test preparation course were significantly different from zero.The mean GMAT scores between second‐time examinees who did and did not use each method of preparation differed inconsistently and only slightly. Additionally, the magnitude of gain over the first administration score was very similar between examinees who used and did not use each method.CONCLUSIONSThis study has indicated that differences in GMAT scores do exist among examinees using different methods of preparing for the examination. However, it was shown that when initial ability, as measured by GMAT first score, was controlled, the sizes of the effects of studying the GMAT bulletin, working through an actual GMAT, and reviewing mathematics were not significantly different from zero. The effects of the methods on GMAT scores of first‐time examinees, for whom a previous score was not available to use as a covariate, were larger. In those analyses (in which self‐reported UGPA was used as a less effective control on ability), the largest effects associated with any method over the other were about 4, 3 and 33 verbal, quantitative and total score points, respectively. However, the effects of using these methods are confounded with the characteristics of examinees who choose to use each method. The effects resulted from a combination of self‐selection and preparation. There do appear to be relationships between method of preparation and test scores. However, it must be emphasized that it does not necessarily follow that using any of the methods of preparation causes an increase in scores.