The Comparison of Effects in 3PL Model and 3PL Testlet Model on the TOEIC

Abstract

Evaluating or measuring test takers’ language abilities is of central importance in the development of standardized language tests that foster meaningful assessment. The Test of English for International Communication(TOEIC) is a widely used standardized test to evaluate test takers’ receptive(reading and listening skills) and productive(writing and speaking skills) language abilities. Although item response theory(IRT) models usually provide satisfactory answers for measuring parameters of items and test takers, they do not always lead to the correct solution when systematic biases are ignored in estimating items and test takers. Due to the paucity of validation studies of the TOEIC, the present study only seeks to explore psychometric properties of items in the simulated TOEIC test in the context of IRT and testlet response theory (TRT). In order to accomplish this purpose, this study describes how a set of items, sharing the same contents in the listening comprehension part of the TOEIC, behaves under the two suggested models (3PL and 3PLT) with the test samples who enrolled in the TOEIC classes in the university in South Korea. Of 100 items on TOEIC listening comprehension session, 60 items (10 items for photographs, 30 items for question-response, 30 items for short conversion, and 30 items for short talks session, respectively) are clustered within each testlet which sometimes called item bundle. Results from this study may provide more precise, reliable, and valid information about estimates regarding items and examinees. Later, suggestions for future research are offered.