A study of the effects of special preparation on GRE analytical scores and item types.

Abstract

A brief curriculum of special preparation for the analytical section of the GRE Aptitude Test was developed and administered to a sample of self-selected GRE candidates at a large eastern university. Students' reactions to the materials and procedures used were obtained. The effects of the program on analytical scores were determined by comparing the scores subsequently obtained by the candidates who undertook special preparation with the scores of all the other candidates who took the test on the same date at the university. Analyses revealed an effect on analytical scores that was both practically and statistically significant. The effect stemmed from improved performance on two of the three analytical item types included in the 1977-1981 analytical section. Hypotheses concerning the components of the special preparation that were responsible for the improvement are discussed. A Study of the Effects of Special Preparation on GRE Analytical Scores and Item Types The effects of special preparation on test scores are of interest to test specialists in several respects. First, a test's susceptibility to score changes that result from various interventions may reflect directly on the validity with which the intended construct is measured. For example, tests designed to reflect student achievement are generally more sensitive to certain interventions, even those that are short-term in nature, than are aptitude tests, which are intended to measure abilities or skills that are developed over a longer period of time. For aptitude tests, sensitivity to short-term special preparation of the kind studied here may indicate that a substantial proportion of the variation in test scores is due to differences in examinees' facility with the methods of assessment or familiarity with the format of items, in contrast to differences in the presumably more stable trait that is being measured. Second, if special preparation increases the test score variance due strictly to differential facility in taking the test, then the predictive power of the test may be impaired since it is unlikely that the same method-related variance would underlie any important criterion. And finally, because all candidates may not have equal access to special preparation, a potential for inequity exists, especially for admissions tests, since any benefits may accrue disproportionately to those examinees who have the time and the financial means to obtain special preparation. The Restructured GRE Aptitude Test The analytical section of the GRE Aptitude Test was introduced in 1977 as a result of a major restructuring of the test (Miller & Wild, 1979). The analytical portion of the test is intended to measure analytical reasoning abilities that, like the verbal and quantitative skills measured by the test, are assumed to develop over a relatively long period of time. The item types originally used in the analytical section of the test are described briefly as follows: 1. Analysis of Explanations: A fact situation paragraph and a surprising result, given the situation, are presented. Approximately 10 numbered statements follow. Each statement is to be evaluated separately from the other statements as to whether it: (A) contradicts something in the fact situation, the result, or both taken together; (B) presents a possible adequate explanation of the result;