COMPARISON OF TRADITIONAL AND ITEM RESPONSE THEORY METHODS FOR EQUATING TESTS

Abstract

Achievement test batteries are typically published in several parallel forms with different levels for different grades. The forms and levels of each different test of the battery must be equated, that is, every score on a given form or level must be translatable into a score value on any other form or level of that test. Equipercentile and linear methods have been used traditionally to equate tests (Angoff, 1971), but item response (latent trait) theory methods have been recently advocated as a potential improvement over traditional methods (Lord, 1977; Wright, 1977). Lord (1977) argued from theoretical considerations that traditional equating methods are not appropriate for equating tests of differing difficulty, whereas item response theory methods have the capacity to provide an appropriate equating in this case. Lord's (1977) definition of equating implies that exact equating is possible only when the tests to be equated measure the same unidimensional ability. Achievement tests covering abilities encountered over a range of grades are probably not unidimensional. The usefulness of a score scale is severely limited, however, unless it spans all of the levels for which the battery is intended. It appears necessary, therefore, to attempt equating across levels even when unidimensionality does not hold. The present study was designed to compare the end results of two traditional equating schemes and seven item response theory equating schemes. The data used came from the 1978 equating project of the Iowa Tests of Educational Development (ITED). The study entailed both the equating of forms of similar difficulty and the equating of levels of differing difficulty. A cross-validation group was used to establish a criterion for comparing the results of different equating methods. A cross-validation summary statistic was calculated which measured the closeness of converted score distributions for stratified randomly equivalent groups. The goal of the study was to identify the method or methods which were best according to this criterion and to examine idiosyncracies of the equating schemes for the equating of a two-level high school achievement test battery.