AN INVESTIGATION OF THE FEASIBILITY OF APPLYING ITEM RESPONSE THEORY TO EQUATE ACHIEVEMENT TESTS1,2,3

Abstract

ABSTRACTThe purpose of this study was to examine the feasibility of using item response theory (IRT) procedures to equate different forms of three achievement tests (Biology, American History and Social Studies, and Mathematics Level II) that are administered by the College Board Admissions Testing program and one achievement test (Advanced Biology) that is part of the Graduate Record Examinations Achievement Test battery. All of the tests investigated in this study are typically equated using conventional linear or curvilinear (equipercentile) methods.Because the investigators had access to data from large scale testing programs that administer many different forms of a test, it was possible to design the study such that a unique criterion for evaluating the equating results, scale drift, was available. Scale drift will occur if the result of equating, say, Form A to Form C through Form B is not the same as that obtained by equating Form A directly to Form C. This study was designed so that for all the Achievement tests investigated, the drift in each scale when IRT equating was used could be compared to the drift occurring when the conventional linear and equipercentile equating methods, used operationally to put new forms on scale, were applied.The results of the study indicate, using scale drift as the evaluative criterion, that it is feasible to use item response theory to equate the four Achievement tests selected for investigation. The results also indicate that the conventional linear methods typically used to equate the tests perform quite adequately. The question of whether the IRT procedures used in the study are sufficiently robust to violations of the assumption of unidimensionality, or whether Achievement tests, of the type used in this study, give rise to sufficiently unidimensional data, must be resolved before the results of the study can be generalized to other Achievement testing situations.