Post-hoc IRT equating of previously administered English tests for comparison of test scores

Abstract

Despite growing concerns about declining scholastic abilities of Japanese students throughout Japan prior to the implementation of the revised Courses of Study in 2002, little empirical evidence was available at that time to support this perceived decline in academic performance. This research describes post-hoc IRT equating of previously administered English tests for retrospective comparison of test scores. Eight forms of a prefecture-(state-)wide English test for high school students administered from 1995 to 2002 were analyzed through the two-parameter IRT model. Because there were no common items and no common examinees among the eight forms, seven composite tests were created for equating with a common-examinee design. Using the mean and sigma method to estimate equating constants, the ability parameter estimates of approximately 140,000 examinees became comparable on a common IRT scale. The results showed that the mean scores among first-year high school students were in a declining trend during the eight-year period. This is a first step toward demonstrating trends in English ability among Japanese students year by year based on IRT using previously administered large-scale English tests. The post-hoc equating method presented here is applicable to any similar circumstance in language testing research.