Item Analysis of Test of Proficiency in Korean: Classical Test Theory and Item Response Theory

Abstract

ABSTRACT This study aimed to examine the quality of both individual items and overall test construction of the Test of Proficiency in Korean (TOPIK) by applying classical test theory and item response theory. The item-level responses of 30 listening items and 30 reading items in each of the three different levels of test from the 28th TOPIK administration were analyzed. Item difficulty and item discrimination parameters were estimated based on classical test theory (CTT) and item response theory (IRT). The item characteristic curves, item information functions, and test information functions based on IRT were also examined. The findings suggest that most items were found to be satisfactory in item difficulty and discrimination, with high correlation between the results analyzed by CTT and IRT. However, the analysis of the IRT-based test information functions revealed that the TOPIK is useful in providing information regarding the examinees with an ability level in only a considerably narrow range. The tests did not provide sufficient information regarding examinee groups who had a high ability or low ability. The findings of this study offer implications in advancing the test by adopting the IRT-based approaches that can provide more information about examinee groups spanning a wide range of abilities so that it can better serve the assessment purpose.