Effects of Test Preparation on the Validity of a of a Graduate Admissions Test

Abstract

Test score improvement has been the major concern in nearly all the extant studies of special preparation, or "coaching," for tests. Recently, however, logical analyses of the possible outcomes and implications of special test preparation (Anastasi, 1981; Cole, 1982; Messick, 1981) have suggested that the issue of test score effects is but one aspect of the controversy sur rounding coaching; the impact of special preparation on test validity is an equally germane consideration. Although the assumption is sometimes made that coaching can serve only to dilute the construct validity and impair the predictive power of a test, some kinds of special preparation may, by reducing irrelevant sources of test difficulty, actually improve both con struct validity and predictive validity. This study ex amined the relationships of both internal and external criteria to Graduate Record Examination (GRE) candi dates' performance on several analytical ability item types, obtained under several test preparation condi tions. The purpose was to assess the effects of these various preparations on test reliability and validity. The preparation conditions were those previously shown to be effective, in varying degrees, in improv ing examinee performance on two of three analytical item types (Powers & Swinton, 1982, 1984). The data for this study were those collected by Powers & Swin ton (1982, 1984). The results suggest that GRE ana lytical ability scores may relate more strongly to aca demic performance after special test preparation than under more standard conditions and that they may re late less to measures of other cognitive abilities (ver bal and quantitative scores). No consistent effects were detected on either the internal consistency or the convergent validity of the analytical measure.