Comparison of Finite State Score Theory, Classical Test Theory, and Item Response Theory in Scoring Multiple-Choice Items

Abstract

Ability estimates obtained from applying finite state score theory, item response models, and classical test theory to score multiple-choice items were compared and contrasted. Responses of 1,230 subjects to the multiple-choice portion of a school-leaving reading comprehension examination were used. Correlations among the ability estimates were nearly perfect. Furthermore, the mean absolute difference values between finite state scores and ability estimates based on the one- and two-parameter models were smaller than that between finite state scores and the ability estimates based on the three-parameter model. The mean absolute difference value between finite state scores and classical test scores was the largest. However, whereas the item response theory and classical test score theory analyses provide item-level information, that is not the case for finite state score theory.