Item Response Theory in Language Testing

Abstract

Language test tasks and items are particularly susceptive to factors that make responses by test candidates ambiguous. Such factors include random error, guessing, compromised test security, and systematic bias. Test analysis technology has evolved since the 1970s to reduce the influence of such construct‐irrelevant factors on test score interpretation. The focus of the present chapter is to introduce different options for undertaking item analysis, with particular focus on item response theory. The different options afforded to language testers are outlined and demonstrated with a working example taken from an authentic foreign language test. A subset of reading comprehension items is analyzed with the use of a classical test theory item analysis approach, which is contrasted with Rasch, two‐parameter IRT, and three‐parameter IRT models. Emphasis is placed on discussion of different measurement assumptions and fit criteria each approach entails, and how item deletion based on each of the four approaches potentially impacts score interpretation and the rank ordering of test candidates.