THE EFFECTS ON IRT AND CONVENTIONAL ACHIEVEMENT TEST EQUATING RESULTS OF USING EQUATING SAMPLES MATCHED ON ABILITY1,2

Abstract

ABSTRACTResearch by Cook, Eignor, and Taft (1988) has clearly demonstrated the need for equating certain achievement tests using samples of students who take the new and old forms at comparable points in the school year. In their study, Cook et al. equated two forms of an achievement test in Biology using two different sample combinations, a spring new‐form/fall old‐form combination and a fall new‐form/fall old‐form pairing. Equating results were quite different across the two sample combinations and all equatings, based on a variety of methods, done using the spring new‐form/fall old‐form combination provided unacceptable results. However, results from recently completed research by Lawrence and Dorans (1988) suggests that matching samples of differing ability (such as the spring new‐form and fall old‐form samples in the Cook et al. study) on an available covariate prior to equating may improve equating results.The purpose of the present study was to examine the effects on IRT and conventional equating results of matching the Cook et al. spring new‐form and fall old‐form Biology samples using two different covariate measures: 1) observed scores on an internal common item equating block; and 2) responses to selected questions on the Student Description Questionnaire (SDQ), a questionnaire that candidates respond to when filling out their registration forms for the test. These matched sample equatings were then compared to the unmatched spring new‐form/fall old‐form equating results and to the equatings based on the samples of comparable ability, the fall new‐form/fall old‐form sample combination.Results of this study indicate that: 1) matching on a set of common items provides greater agreement among the results of the various equating procedures studied than if matching does not take place; and 2) for all equating procedures, the results of the matched group (on the common items) equating agreed more closely with the fall new‐form/fall old‐form criterion equatings than did the fall/spring unmatched equating results. An explanation for why the matching on SDQ responses provided unacceptable equating results is presented in the paper. Finally, the results of this study are compared to the results of other recently completed studies that have also dealt with the matching process.